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ABSTRACT 

Our  computational  experience  with  Gomory's  algorithm  for  the 
integer  linear  programming  problem  of  synthesis  of  optimum  network  with 
NOR  gates  is  presented.   The  problem  is  briefly  described  and  accompanied 
with  statistics  such  as  the  size  and  density  of  the  coefficient  matrix. 

Upon  successful  solution  of  problems  with  90  variables  and  2^0 
rows,  the  effect  of  constraint  orderings  and  adding  additional  inequalities 
was  investigated.   The  difference  in  convergence  of  two  of  Gomory's  pivot 
selection  rules  is  noted.   Also  the  behavior  in  convergence  of  feasible 

versus  non-feasible  problems  is  demonstrated. 

Rn 
The  convergence  rate  is  conjectured  to  be  A'10   where  n  is  the 

number  of  variables  of  a  switching  functi  on  to  be  synthesized  and  A,  B  are 

constant  coefficients.  This  rate  is  compared  to  an  exhaustive  method 

developed  by  Hellerman  to  solve  the  same  logical  design  problem. 
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1 .   INTRODUCTION 

Our  computational  experiments  were  performed  using  Gomory's  all- 
integer  integer  linear  programming  method  even  though  our  variables  are 
restricted  to  be  1  or  0.   The  problems  are  unusual  in  that  the  number 
of  inequalities  is  larger  than  the  number  of  variables .   This  is  just 
the  opposite  of  the  vast  majority  of  the  other  published  computational 
reports.   Also  the  size  of  the  coefficient  matrix  is  much  larger  -  up 
to  90  variables  and  2^0  constraints.   The  problem  formulation  is  derived 
from  the  design  of  logic  circuits  in  digital  computers.   Specifically 
it  is  concerned  with  the  optimum  synthesis  of  a  NOR  element  network 
complicated  by  considering  fan-in  and  fan-out  restrictions.   In  the  next 
section  the  set  of  inequalities  will  be  stated  and  briefly  explained. 

Previous  publications  have  demonstrated  the  very  erratic  be- 
havior in  convergence.   Some  problems  have  been  solved  in  only  a  few 
iterations  while  others  needed  several  hundred  thousand  iterations  for  a 
solution.   '      The  convergence  rate  of  Gomory's  method  seems  to  be 
highly  dependent  upon  the  characteristic  of  the  constraints  of  each 
particular  problem. 

We  will  examine  the  rate  of  convergence  and  the  effect  that 
certain  factors  have  upon  it.   Some  of  the  factors  we  will  consider  are 
the  order  of  the  inequalities,  the  addition  of  other  constraints,  and 
the  difference  between  feasible  and  non-feasible  solutions. 


2.   PROBLEM  STATEM 

Fig.  1:   Feed  Forward  NOR  Circuit  For 
the  Boolean  Function  f(x, ,  x  ,  x  ) 


Our  integer  problem  arises  from  the  attempt  to  synthesize  a 


Boolean  function  f  of  three  variables,  x. ,  x  ,    and  x„  with  a  feed  forward 
NOR  element  circuit  as  is  shown  in  figure  1.    In  the  figure  we  denote 
the  weight  of  an  input  variable  x.  to  the  j-th  NOR  element  by  w.   and  the 
weight  of  the  output  from  the  k-tn  NOR  element  to  the  e-th  element  by 
a,  .   The  logical  design  problem  is  to  determine  which  a' s   and  w's  are  1 
(connected)  and  which  are  zero  (disconnected)  by  using  Gomory's  all-integer 
algorithm.   It  is  possible  that  the  given  number  of  elements  R  is  in- 
sufficient for  a  particular  function,  i.e.  the  problem  is  infeasible. 
Therefore  we  may  have  a  considerable  number  of  infeasible  problems  if 
R  is  small . 


For  a  detail  description  of  the  general  feed  forward  network  synthesis 
formulation  of  which  this  problem  is  a  special  case  see  reference  (8). 
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For  a  detailed  derivation  of  these  equations  see  reference  (8) 
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for  J  =  1,  2,  ...,  8 


For  all  R  elements 

G :  all  wk ' s,  a  ' s,  a.  ^^'s.  andP/^'s  <1 
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These  seven  groups  of  constraints  A-F  compose  the  complete  set 
of  inequalities.   The  constant  U  is  a  sufficiently  large  positive  value  so 
that  if  P,     =  0  the  j -th  inequality  of  A   is  always  satisfied  and  if 


-(j)  .  ,-_  (j)   .  (o)   „  (j), 


=  1  the  j -th  inequality  of  B  is  always  satisfied.   The  input  vector 


X 


[x        ',    xn   ' ,  x  "  ' )    for  the  three  variable  Boolean  function  f(x) 


\1    '   2    '   3 


assumes  all  eight  possible  combinations  of  1  and  0.   Inequalities  D  and 
F  make  the  further  restrictions  that  no  NOR  element  can  have  more  than  3 
inputs  or  more  than  3  outputs.   The  fan-in,  fan-out  constraints  (inequalities 
D  and  F)  arise  from  practical  engineering  restrictions  on  the  actual 
electronic  circuits  which  are  used  to  realize  the  NOR  elements. 

Therefore  cur  all -integer  integer  linear  programming  problem  is: 


3 


i   ::    ;r 


:> .  -,: 


2-1    R 


a 


ek 
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under  the  con  c    Lnts 
A   through  G 
By  minimizing  g  we  are  minimizing  the  total  number  of  connections.   The 
procedure  for  determining  a  network  for  a  given  Boolean  function  is  out- 
lined in  figure  2. 


Fig.  2:   Flow  Chart  Of  Procedure 
of  Synthesizing  A  Boolean  Function 


In  this  manner  we  will  obtain  a  network  with  the  fewest  number  of 
NOR  elements  and  with  minimum  connections. 

The  number  of  variables,  inequalities,  entries,  and  non-zero 
entries  in  the  coefficient  matrix  all  increase  with  the  number  of 
elements  R.  Figure  3  shows  this  growth.  Slack  variables  are  not 
included. 


Fig.  3:   Characteristics  Of  The  Coefficient  Matrix 


R 

Matrix  Size 
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Total   No. 

Non-zero* 

"^Non-zero 

1 

11 

3 

33 

18 

55  =  50 

2 

64 

23 

1472 

183 

12.40 

3 

142 

52 

7380 

447 

6.08 

4 

245 

90 

22,040 

810 

3.68 

5 

374 

137 

51,250 

1277 

2A9 

6 

528 

193 

101, 800 

1836 

1.81 

7 

707 

258 

182,200 

2487 

1.37 

This  number  is  for  the  Boolean  function  f(x)=l.   See  the  E  inequal- 
ities for  the  effect  of  other  Boolean  functions. 


3.    OUTLINE  OF  RESULTS 

The  algorithm  was  programmed  in  assembly  language  NICAP  for 
the  ILLIAC  II  computer  at  the  Univer;  :  nois.  The  computer  has  a  1.75 

microsecond  core   cycle  time,  an  8K  -  52  bi  t  word  memory,  and  the 
facility  to  operate  on  13  bit  quarter  words.   All  the  coefficient  matrix 
entries  were  stored  in  the  13  bit  quarter  words.   Therefore  whenever  the 
algorithm  generates  a  number  greater  in  absolute  value  than  ^+096,  the 
computation  terminates  with  overflow, 

Gomory  in  proposing  his  all -integer  integer  program  suggested 
several,  variations  to  the  basic  algorithm.     We  used  the  variation 
which  chooses  the  pivot  row  which  minimizes  the  lexiograpbic  rank  of 
the  pivot  column.   In  order  to  ascertain  the  factors  important  in 
convergence  of  the  algorithm,  we  solved  several  problems.   These  same 
problems  were  then  reformulated  by  changing  the  order  of  inequalities, 
by  adding  additional,  constraints,  by  changing  the  value  of  U,  etc.   In 
our  tests  the  number  of  problems  run  was  not  large  enough  to  guarantee 
statistical,  accuracy.   This  is  especially  true  since  convergence  if 
very  unpredictable.   However  the  results  do  demonstrate  the  qualitative 
tendency  of  integer  programming. 

The  integer  linear  programming  formulation  with  R  =  M  models 
a  feed-forward  network  of  M  NOP  elements   If  a  Boolean  function  is 
realizable  with  M  or  fewer  elements,  the  corresponding  integer  linear 
program  has  an  optimal  feasible  solution.   But  if  a  function  requires 
more  than  M  elements,  the  problem  is  not  feasible. 

The  set  of  Boolean  functions  upon  which  we  performed  our  exper- 

(1?) 
iment.p  is  shown  in  figure  4,  Hellerman's  "    network  numbering  system  was 

used  to  uniquely  identify  each  function  and  will  be  used  throughout  this  report. 

( 


Fig.  k:      Complete  Set  of  Boolean  Test  Functions 
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9 
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k 

8 
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k 
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The  density  of  non-zero  coefficients  in  the  coefficient 
matrix  after  each  iteration  is  also  of  interest.   The  density  of  the 
initial  matrix  is  low  (see  figure  3)-   In  a  seemingly  typical  problem 
the  density  starts  at  5$  and  increases  gradually  until  it  reaches  about 
15$.   It  oscillates  between  10$  and  20$  until  convergence  at  which  time 
it  drops  down  to  about  10$, 

First  we  noticed  the  size  of  U  in  equations  A  and  B  con  - 
siderably  effects  the  occurrance  of  overflow.   Generally  the  larger 
the  value  of  U  the  more  often  overflow  was  encountered.   By  reducing 
the  value  of  U  from  9  down  to  h   we  were  able  to  prevent  overflow 
from  occurring  as  early.   This  is  especially  true  for  problems  which 

require  a  large  number  of  iterations, 

(1) 

Gomory  also  suggested  another  approach    which  he  called 

the  row  combination  method.   Using  this  method  on  a  set  of  Boolean 
functions  for  R  =  3-   The  comparison  between  the  minimum  rank  method 
and  the  minimum  rank  method  appended  with  the  row  combination  technique 
is  seen  in  figure  5, 


Fig.  5:   Evaluation  of  Adding  Row 
Combination 


Row  combination 

Without  row  combination 

Ave,  number  of  iter, 

62,1 

6-,6 

Time  per  iteration 

46^  msecs 

166     rrsecs 

$  of  problems  with 
overflow 

17 .  6$ 

0.0$ 

Thus  in  considering  the  increased  computation  for  each  iteration,  the 
row  combining  technique  should  be  disregarded. 
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h.      FEASIBLE  AND  NON-FEASIBLE 

For  example,  when  a  Boolean  function  realizable  with  R  =  k 
{h   NOR  elements)  is  tried  with  th<  R  =  3  formulation,  the  resulting 
integer  linear  program  is  not  feasible.   In  Figure  6  the  average 
number  of  iterations  for  all  Boolean  test  functions  and  for  three 
different  orderings  of  the  constraints  are  shown, 

Fig.  6:   Average  No,  of  Iterations 
For  3  Different  Orderings  of  the 
Inequalities „ 


1 

2 

3 

Feasible 

72,5 

66.0 

66.6 

Non- feasible 

85,5 
,  ., j 

83*3 

98,2 

,     1 

The  difference  in  the  average  number  of  iterations  for  any 
case  is  not  too  great.   However  in  the  non-feasible  cases  the  number  of 
iterations  deviates  from  the  average  much  more  than  in  the  feasible  case, 
For  example  in  the  ordering  of  column  3  of  figure  6  all  non-feasible 
functions  required  less  than  92  iterations  ex-  ept  for  #17  and  fflQ   of  figure 
k   which  require  k6l   and  296  iterations  respectively,  The  convergence 
varies  over  a  much  larger  number  of  iterations  in  non -feasible  problems 
than  in  feasible  problems, 

Figure  6  also  demonstrates  '        feasible  problems  are  much 
more  difficult  to  solve  than  the  feasible  problems, 


See  figure  7  for  details  of  the  constraint  ordering 
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5.   ORDER  OF  INEQUALITIES 

Gomory's  method  with  the  incorporation  of  the  lexiographic 

(1) 
row  rank  variation     is  sensitive  to  the  order  of  inequalities  since 

the  lexiographic  ordering  is  changed  by  altering  the  order  of  the 

constraints.   Figure  7  displays  the  average  number  of  iterations  for  all 

the  functions  of  figure  h   for  four  different  constraint  orderings . 

The  objective  function  is  denoted  by  O.F.  and  slack  variables  always 


Fig.  7:   Average  Number  of  Iterations 
Under  k   Different  Constraint  Orderings 
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A3 

F 

E 

Ai 

C3 

A3 

A3 

** 

A2 

C3 

C3 
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66.6 
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98.2 

118.0 
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appear  after  all  other  constraints.  It  should  be  noted  that  B  }  B  , 
and  E  are  the  only  constraints  which  initially  have  negative  entries 
in  the  constant  column. 

It  is  difficult  to  draw  any  conclusive  results  from  figure  7 • 
However,  it  seems  best  to  place  the  E  constraints  first  since  we  have 
more  non-feasible  than  feasible  problems.   These  inequalities  are  the  only 
ones  which  change  according  to  the  Boolean  function  which  is  being  tested. 
Since  the  rest  of  the  constraints  remain  the  same  regardless  of  the 
Boolean  function  being  solved,  the  E  constraints  are  likely  the  most 
important.   Therefore  we  place  them  at  the  top.   One  of  the  reasonable 
orderings  may  be  to  put  the  more  important  inequalities  at  the  upper  part 
of  the  tableau.   However,  strictly  speaking  we  don't  have  any  measure 
of  the  importance  of  an  inequality,   One  intuitive  and  reasonably 
successful  measure  is  to  count  the  number  of  non-zero  entries  in  an 
inequality  -  the  greater  the  number  of  coefficients  the  more  important 
the  constraint. 
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6.   ADDITIONAL.  CONSTRAINTS 

Another  technique  for  increasing  the  speed  of  convergence  is 

to  incorporate  additional  constraints  which  exclude  unnecessary  solutions 

without  loss_of  generality.   For  example  the  two  networks; 
x 


realize  the  same  Boolean  function.   But  it  is  not  important  to  get  both 
solutions.   Either  one  will  be  acceptable.   Therefore  additional  constraints 
should  be  added  to  exclude  either  one  but  obviously  not  both. 

By  considering  other  properties  of  a  h   element  NOR  network 
(R  =  k-)   we  establish  the  following  constraints; 
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Note  that  some  of  the  inequalities  require  that  the  network  consist  of 
exactly  4  elements  while  others  require  that  the  Boolean  function  have 
exactly  3  input  variables.   The  additional  constraints  for  the  R  =  3 
network  are  obtained  by  neglecting  the  1st  element  and  relabeling  the  re- 
maining elements  2,    3  and  4  as   1,  2,    and  3  respectively. 

Figure  8  shows  the  effect  of  adding  these  constraints  to  the 
first  3  row  orderings  of  figure  7  for  the  R  =  3  network  and  for  the  six 
Boolean  functions  #5 -#10.   Figure  9  demonstrates  the  improvement  on  the 
R  =  4  network  for  all  the  4  gate  realizable  Boolean  functions,  Without 
the  additional  constraints  all  the  R  =  4  network  problems  either  did 
not  converge  after  about  1200  iterations  or  generated  an  overflow.   Al- 
though all  the  infeasible  R  =  4  problems  tested  with  additional  constraints 
did  not  converge  after  1200  iterations,  the  additional  constraints  were 
extremely  successful,  for  feasible  problems  in  the  R  =  4  case. 

Fig.  8:   Effectiveness  of  Additional  Constraints  on  R  =  3  Network, 
Average  Number  of  Iterations 
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Fig.  9:    Convergence  with  Additional  Constraints  on  R  =  k   Network 
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From  the  results  we  notice  the  fact  that  the  additional 
constraints  facilitate  convergence  if  the  problem  is  feasible  but  make 
convergence  worse  if  it  is  non-feasible,,   Furthermore,  the  incorporation 
of  the  added  inequalities  is  much  more  effective  with  the  R  =  k   problems 
than  the  R  =  3  problems . 


A  few  functions  were  tried,   However  all  exceeded  the  preassigned 
bound  of  1200  iterations . 
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7.   FIXING  THE  VALUES  OF  A  VARIABLE 

We  tried  splitting  the  given  problem  into  two  smaller 
problems  by  fixing  a  particular  variable  to  1  in  one  and  to  0  in  the 
other.   By  comparing  the  results,  we  can  pick  the  optimal  network.   By 
picking  a,  _  in  "the  R  =  3  network  and  by  using  the  formulation  of  column 
h   of  figure  7  we  obtained  the  results  of  figure  10. 

Fig.  10:   Average  Number  of  Iterations  For  All  Test  Functions 
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Generally  it  is  difficult  to  judge  whether  fixing  a  variable 
speeds  up  the  computation.  However,  it  may  be  worthwhile  if  the  right 
choice  of  a  variable  is  made. 
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8.   RELATIONSHIP  BETWEEN  THE  NUMBER  OF  ITERATIONS  AND  VARIABLES 

In  order  to  determine  the  increase  in  the  number  of  iterations 

we  preset  some  randomly  picked  variables  to  their  solution  values. 

(Solution  of  our  problem  is  already  known  by  using  another  approach. ) 

This  equivalently  shrinks  the  size  of  integer  linear  programming 

problems,  but  the  general  characteristics  of  the  problem  may  not  change 

very  much.   The  test  was  performed  on  one  particular  Boolean  function 

ac  v  be  with  the  following  order  of  inequalities 

OF 


A 


E 

C3 

F 
G 

Each  size  of  the  coefficient  matrix  was  run  5  times  with  a  different  set 

of  variables  being  preset.   The  average  number  of  iterations  is  plotted 

in  figure  11. 

Fig.  11:   Number  of  Iterations  l(n)  as  a  Function  of  the  Number  of 
Variables  n  For  the  Boolean  Function  ac  ^  toe. 
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The  curve  of  figure  11  demonstrates  that  the  increase  in  the 
number  of  iterations  grows  exponentially  and  can  he  expressed  as 

I(n)  B  2.5  x  10"02T7n 
for  the  Boolean  function  ac  ^   be. 

Our  conjecture  is  that  the  number  of  iterations  is  generally 

"Rn 
A* 10    where  A  and  B  are  constants  which  depend  upon  the  particular 

type  of  integer  linear  programming  problem. 
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9.   COMPARISON  OF  HELLERMAN "S  ALGORITHM  AND  GOMORY'S 

Hellerman  in  reference  12  determined  the  optimum  NOR  circuit 
for  all  Boolean  functions  f(x,,  Xp,  x  )  by  an  exhaustive  method,,   He 
generated  all  possible  circuits  and  chose  the  best  one   for  each  function. 

It  is  interesting  to  compare  the  integer  programming  approach 
using  Gomory's  method  with  Hellerman' s  to  determine  if  it  is  better.   For 
the  3  and  k   element  formulation  (R  =  3>  M  the  computational  efficiency 

of  Gomory's  method  is  inferior  to  Hellerman 's  approach,   The  number  of 

2 
CR 
iterations  in  Hellerman 's  method  increases  as  T-10    with  the  number 

of  elements  R. 

In  the  previous  section  we  showed  that  Gomory's  method  appar- 

ently  increases  as  A-10   .   Therefore  it  is  unlikely  that  Gomory's 

algorithm  would  be  better  than  Hellerman 's  exhaustive  method  since  n 

2 
increases  as  R  .   However,  this  cannot  be  a  definite  conclusion  since 

we  do  not  have  sufficient  computational  experience.   Also  some  techniques 

to  reduce  the  number  of  iterations  for  Gomory's  method  may  be  developed 

in  the  future  which  could  make  it  more  effective  tha.n  Hellerman' s 

approach. 
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10.   CONCLUSIONS 

Using  Gomory's  algorithm  ve  duci  es  i'ully  solved  a  few  rather 
large  0-1  value  all  integer  problems  of  90  variables  and  2^0  constraints 

Gomory's  work  was  followed  by  the  implicit  enumeration 
methods  of  Balas^   ,  GeoffriorT  '    and  others^  '    '         .  These 
algorithms  further  restrict  the  problem  by  requiring  all  variables  to 

be  1  or  0.   The  number  of  iterations  for  implicit  enumerations  reportedly 

k   (ID 

grow  as  n  . 

Since  our  problems  are  0-1  problems,  we  are  now  experimenting 
with  these  implicit  enumeration  methods.   We  are  very  encouraged  with 
the  initial  results  and  feel  these  methods  may  be  better  for  our 
problems.   Our  results  will  be  published. 
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